A computer printout is attached hereto as an appendix in microfiche form and is incorporated herein by reference. The printout comprises executable program files in hexadecimal format. This appendix includes 2 microfiches, containing a total of 138 frames.
The present invention relates generally to network data communications, and specifically to real-time multimedia broadcasting over a network.
In network broadcasting, data are transmitted over a network in real time from a single transmitting computer to a plurality of clients simultaneously. The network may be a LAN, a WAN, an intranet or a public network such as the Internet. Network broadcasting is most commonly used to stream multimedia data, typically comprising images and sound.
FIG. 1 is a schematic illustration showing a real-time broadcasting system 20, as is known in the art. One or more input devices 22 (for example, a video camera and/or microphone) are used to generate a multimedia data stream representing an entertainment or informational program to be transmitted to a plurality of clients 30 via a network 28. Because of bandwidth limitations of the network, the data stream from host 22 must first be compressed by a real-time encoder 24 and then routed to appropriate clients 30 by a broadcast server 26 (since not all clients on the network are necessarily intended to receive the broadcast).
Encoder 24 and server 26 typically comprise high-cost, dedicated computer systems, such as a Sun Station (produced by Sun Microsystems) or a Windows NT server, running suitable RealSystem 5.0 software (produced by RealNetworks Inc., Seattle, Wash.). These dedicated systems are required in order to ensure that the data stream is distributed and received by clients 30 in real time. Similarly, host 22 must typically be connected directly to encoder 24 by a high-speed data link or LAN, and not via the Internet or other narrowband network. Therefore, real-time broadcasting is normally possible only for hosts having a suitable, dedicated encoder and broadcast server and cannot be offered by Internet service providers (ISPs) to their general clientele.
It is an object of some aspects of the present invention to provide substantially continuous, high-bandwidth data streaming over a network using common, existing server and network infrastructure.
It is a further object of some aspects of the present invention to provide data broadcasting capability, particularly for multimedia data, without the need for a dedicated broadcast computer system.
It is a further object of some aspects of the present invention to provide apparatus and methods for data broadcasting at reduced cost by comparison with systems known in the art.
It is still another object of some aspects of the present invention to enable a personal computer to remotely broadcast a multimedia program through an Internet service provider (ISP) using common, universally-supported Internet communication protocols.
In preferred embodiments of the present invention, a transmitting computer generates a data stream and broadcasts the data stream via a network server to a plurality of clients. The data stream is divided into a sequence of segments or slices of the data, preferably time slices, wherein the data are preferably compressed. Each slice is preferably assigned a respective slice index. The transmitting computer uploads the sequence of slices to the server substantially in real time, preferably using an Internet protocol, most preferably the File Transfer Protocol (FTP), as is known in the art. The clients download the data stream from the server, preferably using an Internet protocol, as well, most preferably the Hypertext Transfer Protocol (HTTP), or alternatively, using other protocols, such as UDP or RTP, which are similarly known in the art. The clients use the slice indices of the frames to maintain proper synchronization of the playback. The division of the data stream into slices and the inclusion of the slice indices in the data stream to be used by the clients in maintaining synchronization allows the broadcast to go on substantially in real time without the use of special-purpose hardware.
Preferably, each segment or slice is contained in a separate, respective file. Alternatively, the segments or slices may all be contained in a single indexed file, which is streamed to the client in a series of packets, each covering a range of one or more indices. HTTP version 1.1 supports this sort of file streaming. Other protocols may also be used for this purpose.
In some preferred embodiments of the present invention, the data stream comprises multimedia data captured or generated by the transmitting computer. The term xe2x80x9cmultimediaxe2x80x9d as used in the context of the present patent application and in the claims refers to images or sound or to data representative of images or of sound or a combination thereof. Multimedia image data may include still images, video, graphics, animation or any combination thereof, including text displayed in conjunction therewith. It will be appreciated, however, that the principles of the present invention may similarly be applied to streaming of other data types.
Preferably, the transmitting computer compresses the frames in the data stream, most preferably using methods of image and audio compression such as those described in U.S. patent application Ser. No. 08/919,027, which is assigned to the assignee of the present patent application and incorporated herein by reference. Alternatively, any suitable methods of compression known in the art may be used. The compressed data are conveyed to the server and thence to the clients, which decompress the data.
In some preferred embodiments of the present invention, the transmitting computer and the clients monitor the uploading and downloading of data to and from the server, respectively, in order to determine the amount of time required to convey each slice and to verify that the slices are conveyed at a sufficient rate. When the data stream comprises multimedia data, the data rate should be generally equal to or faster than the rate at which the data are generated at the transmitting computer.
In some of these preferred embodiments, the transmitting computer and/or the clients each open a plurality of FTP or HTTP links, respectively, with the network server. The slices are transferred over different ones of the links in alternation. Although typically none of the plurality of links has sufficient bandwidth on its own to convey the entire data stream in real time, the combined bandwidths of the plurality of links are generally sufficient for this purpose. Preferably, each of the links is monitored to determine its specific data transfer rate. If the transfer rate of any of the links is below a predetermined minimum, that link is preferably closed, and a new link is opened in its place.
In other preferred embodiments, the slices are provided by the server at multiple resolution or quality levels. Each such level has a different degree of data compression, and thus corresponds to a different data bandwidth requirement. The client or the server monitors the data transfer rate of a data link opened therebetween and selects the level that is appropriate to the link bandwidth. If the monitored data transfer rate changes during transmission, the quality level is preferably reselected accordingly.
Preferably, the transmitting computer monitors the bandwidth of the data stream that it is uploading to the server, and compares the data stream bandwidth to a known or estimated bandwidth of the link or links between the transmitting computer and the server. The transmitting computer preferably compresses the data stream at a compression ratio that is adjusted so as to match the data stream bandwidth to the available link bandwidth, using methods described, for example, in the above-mentioned U.S. patent application Ser. No. 08/919,027.
There is therefore provided, in accordance with a preferred embodiment of the present invention, a method for real-time broadcasting from a transmitting computer to one or more client computers over a network, including:
providing at the transmitting computer a data stream having a given data rate;
dividing the stream into a sequence of slices, each slice having a predetermined data size associated therewith;
encoding the slices in a corresponding sequence of files, each file having a respective index; and
uploading the sequence to a server at an upload rate generally equal to the data rate of the stream, such that the one or more client computers can download the sequence over the network from the server at a download rate generally equal to the data rate.
Preferably, dividing the stream into the sequence of slices includes dividing the stream into a sequence of time slices, each having a predetermined duration associated therewith.
Preferably, uploading the sequence includes comparing the upload rate to the data rate and adjusting the upload rate responsive to the comparison. Further preferably, encoding the stream includes compressing data in the stream at a desired compression ratio, and adjusting the upload rate includes changing the compression ratio. Alternatively or additionally, adjusting the upload rate includes adjusting the size of one or more of the slices.
Preferably, uploading the sequence includes opening a plurality of file transfer links between the transmitting computer and the server, each link characterized by a respective link data rate, and uploading different files in the sequence over different ones of the plurality of links. Further preferably, opening the plurality of links includes opening links such that the data rates of the links taken together are sufficient to upload the sequence at the upload rate generally equal to the data rate.
Preferably, uploading the sequence includes uploading a sequence using an Internet Protocol, most preferably using FTP.
Preferably, the method includes downloading the sequence using an Internet protocol, most preferably HTTP, or alternatively, UDP or RTP, over the network from the server to the one or more client computers. Preferably, the one or more client computers decode the sequence and play back the data stream responsive to the indices of the files, at a replay rate generally equal to the data rate.
Preferably, uploading the sequence includes uploading and updating an index file containing the index of the file in the sequence that was most recently uploaded, and the one or more client computers read the index file to play back the sequence. In a preferred embodiment, downloading the sequence includes selecting a file in the sequence earlier than the file whose index is contained in the index file and downloading at least a portion of the sequence of files beginning with the selected file.
Preferably, the one or more client computers include a plurality of client computers, and downloading the sequence includes downloading to the plurality of client computers substantially simultaneously.
Preferably, downloading the sequence includes opening a plurality of download links between one of the client computers and the server, each link characterized by a respective link data rate, and downloading different files in the sequence over different ones of the plurality of links. Most preferably, opening the plurality of links includes opening links such that the data rates of the links taken together are sufficient to download the sequence at the download rate generally equal to the data rate.
Preferably, opening the plurality of links includes monitoring the data rates of the links and opening a new link in place of one of the links having a data rate lower than a predetermined level.
In one preferred embodiment, opening the new link includes retransmitting at least one of the files in the sequence, wherein the at least one of the files was incompletely transmitted over the one of the links having the data rate lower than the predetermined level.
In another preferred embodiment, opening the new link includes dropping at least one file out of the sequence, wherein the at least one of the files was incompletely transmitted over the one or more of the links having the data rate lower than the predetermined level.
In still another preferred embodiment, encoding the slices includes encoding slices at a plurality of different quality levels, such that the files corresponding to a given one of the slices have a different, respective data size for each of the quality levels. Preferably, downloading the sequence includes determining a data bandwidth of the network between the server and the client computer and selecting one of the quality levels responsive to the determined bandwidth.
Preferably, the data stream includes multimedia data.
There is further provided, in accordance with a preferred embodiment of the present invention, apparatus for real-time broadcasting of a data stream having a given data rate over a network, including:
a transmitting computer, which divides the stream into a sequence of slices, each slice having a predetermined data size associated therewith, and encodes the slices in a corresponding sequence of files, each file having a respective index, and
which uploads the sequence to a server at an upload rate generally equal to the data rate, such that one or more client computers can download the sequence over the network from the server at a download rate generally equal to the data rate.
Preferably, the transmitting computer compares the upload rate to the data rate and adjusts the upload rate responsive to the comparison. Most preferably, the transmitting computer compresses the data at a compression ratio which is varied responsive to the comparison. Additionally or alternatively, the transmitting computer adjusts the size of one or more of the slices responsive to the comparison.
Preferably, the transmitting computer opens a plurality of links between the transmitting computer and the server, each link characterized by a respective data rate, and transmits different ones of the sequence of files over different ones of the plurality of links. Most preferably, the transmitting computer opens the plurality of links such that the data rates of the links taken together are sufficient to upload the sequence at the upload rate generally equal to the data rate. Further preferably, the transmitting computer monitors the data rates of the links and opens a new link in place of one of the links whose data rate is lower than a predetermined level.
In a preferred embodiment, the slices are encoded at a plurality of different quality levels, such that the files corresponding to a given one of the slices have a different, respective data size for each of the quality levels.
Preferably, the transmitting computer uploads the sequence using an Internet upload protocol, most preferably FTP.
Preferably, the one or more client computers decode the sequence and play back the data stream responsive to the indices thereof, at a data replay rate generally equal to the data rate. Preferably, the one or more client computers download the encode sequence using an Internet download protocol, most preferably HTTP or alternatively, UDP or RTP.
Preferably, the one or more client computers include a plurality of client computers, which download the sequence substantially simultaneously.
Preferably, the network includes the Internet.
Further preferably, the data stream includes multimedia data, and the predetermined data size of each of the slices corresponds to a time duration of the slice.
The present invention will be more fully understood from the following detailed description of the preferred embodiments thereof, taken together with the drawings in which: